A model driven architecture for adaptable overlay networks

Recent years have witnessed a remarkable spread of interest in decentralised infrastructures for Internet services. Peer-to-Peer systems and overlay networks have given rise to a major paradigm shift and to novel challenges in distributed systems research. Numerous projects from several research communities and commercial organisations have started building and deploying their own systems. Adoption, however, has been restricted to sparsely selected areas, dominated by few applications. Among the technical reasons for the limited availability of deployable systems is the complexity of their design and the incompatibility of systems and frameworks. Applications are tightly coupled to a specific overlay implementation and the framework it uses. This leaves developers with two choices: implementing their application based on the fixed combination of overlay, networking framework and programming language, or reimplementing the overlay based on the desired application framework. Both approaches have their obvious draw-backs. Implementing an overlay, even as a reimplementation, is a task that exhibits considerable challenges. Protocols have to be adapted, completed and partially reverse engineered from the original system to implement them correctly. Message serialisations and interfaces have to be rewritten for the new framework. State maintenance and event handling are programmed in very different ways in different environments, which typically requires their redesign. Reimplementing an overlay for a new environment is therefore not necessarily less work than designing a new one. As for the alternative, being tied to a specific environment prevents the application designer from freely choosing the best suited framework for the specific application. Deploying different overlays in one application is only possible if they were written for the same framework, and even then, running multiple non-integrated overlays at the same time can become prohibitively resource extensive. Testing with different overlay topologies and adapting to different deployment environments is similarly hard in this scenario. The current techniques used for overlay implementation turn out to become limiting factors in the design of overlay applications. The approach taken by this thesis tackles these issues at design time, at a point long before integration problems arise. It presents a modelling framework that allows to express overlay specific semantics in a platform-independent, domain specific language called the Overlay Modelling Language, OverML. A Model Driven Architecture maps these abstract overlay specifications to concrete, framework specific implementations. Applications based on OverML benefit from state sharing between different overlays and from short topology implementations in the SQL-Like Overlay Specification Language SLOSL. Their conciseness allows an easy adaptation of SLOSL statements to specific quality-of-service requirements. The architecture provides a clean separation between the generated implementation and handwritten components through generic, event-driven interfaces. To evaluate the approach, a series of topologies is exemplarily specified in OverML and/or SLOSL. A complete walk-through from the specification to the deployment of an OverML implemented overlay is additionally presented. The major contribution of this thesis is the first complete design methodology for the integrative, high-level development of portable, adaptable overlay networks.